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Abstract.  We  consider  a  communications  network  consisting  of  many  mobiles. 
There  are  random  external  data  processes  arriving  at  some  of  the  mobiles,  each  destined 
for  a  unique  destination  or  set  of  destinations.  Each  mobile  can  serve  as  a  node  in 
the  possibly  multi-hop  (and  not  necessarily  unique)  path  from  source  to  destination.  At 
each  mobile  the  data  is  queued  according  to  the  source-destination  pair..  Time  is  divided 
into  small  scheduling  intervals.  The  capacity  of  the  connecting  channels  are  randomly 
varying.  The  system  resources  such  as  transmission  power  and/or  time,  bandwidth, 
and  perhaps  antennas,  must  be  allocated  to  the  various  queues  in  a  queue  and  channel- 
state  dependent  way  to  assure  stability  and  good  operation.  Lost  packets  might  or 
might  not  have  to  be  retransmitted.  At  the  beginning  of  the  intervals,  the  channels 
are  estimated  via  pilot  signals  and  this  information  is  used  for  the  scheduling  decisions, 
which  are  made  at  the  beginning  of  the  intervals.  Stochastic  stability  methods  are  used 
to  develop  scheduling  policies.  The  resulting  controls  are  readily  implementable  and 
allow  a  range  of  tradeoffs  between  current  rates  and  queue  lengths,  under  very  weak 
conditions.  The  basic  methods  are  an  extension  of  recent  works  for  a  system  with  one 
transmitter  that  communicates  with  many  mobiles.  The  choice  of  Liapunov  function 
allows  a  choice  of  the  effective  performance  criteria.  All  essential  factors  are  incorporated 
into  a  “mean  rate”  function,  so  that  the  results  cover  many  different  systems.  Because 
of  the  non-Markovian  nature  of  the  problem,  we  use  the  perturbed  Stochastic  Liapunov 
function  method,  which  is  designed  for  such  problems.  Various  extensions  (such  as  the 
requirement  of  acknowledgments)  are  given,  as  well  as  a  useful  method  for  getting  the  a 
priori  routes. 
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1.  Introduction.  The  paper  considers  the  problem  of  scheduling  in 
a  network  of  M  mobiles  (to  be  referred  to  as  nodes)  with  time  varying 
link  capacities.  There  are  many  (S)  external  sources  with  bursty  data  pro¬ 
cesses,  each  sending  its  data  to  its  unique  origin  node,  to  be  sent  through 
the  network  to  a  unique  (except  for  the  multicasting  case)  destination  node. 
At  each  mobile,  the  data  is  queued  until  transmitted,  in  an  infinite  buffer 
depending  on  the  source-destination  pair.  Some  mobiles  serve  as  interme¬ 
diaries  in  the  possibly  multi-hop  connections  between  sources  and  desti¬ 
nations.  The  routes  between  source  and  destination  need  not  be  unique. 
We  are  concerned  with  the  efficient  and  stabilizing  allocation  of  the  sys- 
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tern  resources,  say,  transmission  power,  time  and  bandwidth,  to  the  various 
queues  at  each  mobile  in  a  queue  and  channel-state  dependent  way.  Time 
is  divided  into  small  scheduling  intervals.  The  capacities  of  the  connecting 
channels  in  each  interval  form  a  correlated  random  process.  At  the  begin¬ 
ning  of  the  intervals,  the  capacities  (or  surrogates  such  as  the  S/N  ratios) 
are  estimated  where  possible  via  pilot  signals  and  this  information  is  used 
for  the  scheduling  during  that  interval.  The  resource  allocation  decisions 
are  made  at  the  beginning  of  the  intervals.  Owing  to  the  random  nature  of 
the  arrival  and  channel  processes,  the  computation  or  even  the  existence 
of  stabilizing  policies  is  not  at  all  obvious.  The  approach  is  a  network 
extension  of  the  development  for  the  one-node  case  in  [4]. 

The  channel  processes  are  usually  non-Mar kovian.^  Even  if  it  and  the 
arrival  processes  were  Markovian,  it  would  be  extremely  difficult  to  use 
classical  stability  methods,  but  the  versatile  perturbed  Liapunov  function 
method  [4,  7]  can  be  used  to  obtain  stabilizing  scheduling  policies.  Let  X 
denote  the  vector  of  all  queue  values  at  all  of  the  nodes  (all  data  quantities 
are  measured  in  packets).  With  the  perturbed  Liapunov  function  method 
one  starts  with  a  basic  Liapunov  function  V(X)  that  works  for  an  approx¬ 
imating  “mean  flow”  system  where  the  randomness  has  been  averaged  out 
in  a  particular  “controlled”  way.  Then  one  gets  a  perturbation  5V (n)  so 
that  V{X{n))  +  dV{n)  can  be  used  as  a  Liapunov  function  for  the  true 
non-Markov  physical  system.  Analogously  to  the  “stability”  method  for 
selecting  controls,  the  controls  are  determined  by  “approximately”  mini¬ 
mizing  a  conditional  expectation  of  the  rate  of  change  of  the  basic  Liapunov 
function  along  the  random  path.  The  formulas  are  simple  and  the  algorithm 
is  readily  implemented.  For  simplicity,  we  use  a  basic  Liapunov  function 
that  is  a  polynomial  which  is  the  sum  of  terms,  each  depending  on  a  single 
component  of  the  state  of  the  queue.  This  seems  to  be  adequate  for  current 
needs,  but  a  large  family  of  strictly  convex  separable  functions  can  also  be 
used.  The  end  result  is  that,  if  a  certain  “mean  flow”  is  stabilizable,  then 
so  is  the  physical  system  under  our  scheduling  rule.  This  stabilizability 
condition  can  often  be  readily  verified,  and  appears  to  be  very  close  to  a 
necessary  condition. 

Some  useful  extensions  are  discussed  in  Section  4.  There  we  give  the 
modest  changes  that  are  required  when  a  packet  can  be  lost  and  the  re¬ 
ceipts  on  each  individual  link  must  be  acknowledged.  The  multicasting 
case  is  briefly  outlined  and  there  is  a  discussion  of  a  simple  model  where 
the  number  of  sources  can  vary  in  time.  Various  extensions  are  implic¬ 
itly  included  in  the  basic  formulation.  For  example,  channel  breakdowns, 
priority  users,  and  random  connectivity. 

The  {n  +  l)st  scheduling  interval  will  be  called  the  nth  slot.  The 
argument  (n)  denotes  the  beginning  of  the  nth  slot,  and  is  referred  to 
as  “time  n.”  Let  ^(n)  denote  the  queue  size  at  time  n  at  node  k  of 


^E.g.,  Rayleigh  fading. 
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data  coming  from  source  i  (defined  to  be  zero  if  node  k  is  not  on  the 
path  for  source  i).  Define  the  vectors  Xk{n)  =  <  S'}  and  X{n)  = 

{Xk{n),  fc  <  Mj,  with  canonical  values  Xk  and  X,  resp.  With  given  weights 
the  basic  Liapunov  function  will  be^ 

=  (1-1) 
i,k 


A  stability  analysis  should  assure  robustness  of  behavior  to  small 
changes  in  the  process  dynamics;  hence  it  is  preferable  to  use  methods 
that  do  not  require  the  Markov  property.  The  perturbed  Liapunov  func¬ 
tion  method  does  not  require  Markovianness.  In  applications,  there  are 
often  many  criteria  that  are  of  interest,  e.g.,  mean  delay  and  variance  of 
delay.  One  should  experiment  with  the  form  of  the  Liapunov  function  and 
examine  the  effects  of  the  associated  scheduling  rules  in  order  to  get  in¬ 
sight  into  the  tradeoffs  between  competing  criteria.  Such  an  experimental 
procedure  would  give  more  insight  and  better  rules  than  those  obtained 
with  a  single  fixed  rule.  The  wide  choice  of  functions  V{X)  facilitates  such 
experimentation. 

There  is  much  work  on  scheduling  in  the  presence  of  various  types 
of  channel  and  data  process  randomness.  But  very  little  is  available  on 
scheduling  for  the  general  network  case  when  the  channels  are  randomly 
varying  in  a  non-trivial  way.  For  the  one-node  case  where  the  rate  of  trans¬ 
mission  is  proportional  to  power,  [1,  9]  gets  rules  for  power  allocation  whose 
form  is  similar  to  ours  when  p  =  2  (such  rules  are  called  “max  weight” 
there),  and  which  are  based  on  stability  considerations.  The  method  uses 
large  deviations  estimates  and  the  setup  is  Markovian.  See  also  [4].  The 
reference  [11],  perhaps  the  first  to  deal  with  random  channels  in  a  network, 
allocates  power.  Since  their  channel-rate  and  data-arrival  processes  are  all 
i.i.d.  sequences  (this  assumption  is  required  by  their  method),  the  possible 
applications  are  very  limited. 

The  papers  [2,  3]  deal  with  related  problems,  again  essentially  for  one- 
node  systems.  There  is  a  set  of  parallel  processors,  and  the  connectivities 
between  the  sources  and  the  processors  (but  not  the  outgoing  channels) 
vary  randomly.  They  prove  results  concerning  the  limit  (as  f  ^  oo)  of 
(queue  length  at  t)/t,  and  give  conditions  under  which  this  limit  is  zero. 
This  is  used  to  show  that  the  integral  of  the  “rates”  of  transmission  per  unit 
time  converges.  They  allocate  a  single  resource  (e.g.,  bandwidth)  and  the 


^We  could  use  >  where  >  2,  >  0  or  V{X)  = 

X/i  fe  where  the  Vi^ki')  are  strictly  convex  non-negative  functions,  whose 

first  derivative  DVi  ki^i  k)  i®  o(Vi  ki^i  k))  and  second  derivative  is  o(DVi^k{^i  k)-  One 
can  choose  the  function,  for  example,  to  model  upper  bound  constraints  on  some  queues. 
The  choice  of  the  functions  and  powers  allows  a  variety  of  tradeoffs  between  queue  size 
and  throughput.  We  use  (1.1)  since  the  notation  is  simpler.  But  the  development  is 
parallel  for  the  other  cases,  and  the  same  conditions  would  be  used. 
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rate  is  proportional  to  the  allocation.  Our  proof  is  easily  adapted  to  that 
problem,  with  the  definition  of  stability  to  be  used  here.  The  work  [10],  for 
a  one  node  model,  has  a  Markovian  channel-state  process,  the  data  input 
sequence  is  i.i.d.,  and  a  “complete  resource  pooling”  condition  is  required. 
The  decision  rule  is  the  same  as  ours  for  a  quadratic  Liapunov  function. 
The  emphasis  is  on  stability  and  simplification  of  the  model  in  the  heavy 
traffic  limit.  The  paper  [6]  treats  the  same  subject  as  this  paper,  but  the 
routes  are  restricted  to  be  unique,  and  the  set  of  extensions  is  different. 
When  acknowledgments  are  required  (as  in  Section  4),  they  are  sent  to  the 
origin  node.  Here  transmission  on  each  link  must  be  acknowledged.  The 
developments  differ  in  the  type  of  Liapunov  function  perturbations  that 
are  used.  See  also  [5]  for  a  stability  analysis  as  the  heavy  traffic  regime  is 
approached. 


2.  The  Problem  Formulation.  Definitions.  Let  k  denote  a  canon¬ 
ical  node  and  let  (z,  k)  denote  the  queue  of  source  i  data  at  node  k.  Since 
the  routing  is  not  necessarily  unique,  queue  (z,  k)  might  have  possible  for¬ 
ward  links  to  any  number  of  other  nodes.  Let  {/(z,  fc,  a),  a}  denote  the 
possible  next  nodes  for  queue  (z,fc).  These  are  indexed  by  the  parameter 
a,  whose  value  ranges  over  a  set  that  depends  on  z,  k.  This  set  will  not 
be  specified,  but  all  summations  over  a,  for  fixed  z,  k,  are  assumed  to  be 
over  this  set.  Similarly,  queue  (z,  k)  might  receive  data  from  any  number  of 
other  nodes.  Let  {5(z,  fc, /3), /?}  denote  the  possible  nodes  from  which  (z,fc) 
can  receive  data,  indexed  by  the  parameter  (3,  whose  value  ranges  over  a 
set  that  depends  on  i,k.  This  set  will  not  be  specified,  but  all  summations 
over  (3,  for  fixed  z,  k,  are  assumed  to  be  over  this  set.  If  no  route  for  source 
i  uses  node  k,  then  queue  (z,  k)  does  not  exist,  and  we  ignore  Xij^,  f{i,  k,  a) 
and  b{i,k,(3).  If  the  routing  from  (z,  fc)  is  unique,  then  a  takes  only  one 
value.  If  k  is  the  origin  node  for  source  z,  then  terms  involving  b(i,  k,  (3)  are 
ignored,  as  are  terms  involving  f(i,  k,  a)  if  node  k  is  the  terminal  node  for 
source  i. 

Let  Lk{n)  denote  the  (vector)  set  of  channel  states  at  node  k,  at 
time  n.  It  is  a  vector  consisting  of  the  states  of  all  of  the  possible  links 
{(z,  /(z,  k,  a));  z,  a}  that  are  outgoing  from  node  k.  Lk{n)  could  be  just  the 
set  of  S/N  ratios  at  the  receivers  corresponding  to  unit  transmitted  power, 
or  it  might  be  some  other  indicator  of  the  link  capacities.  It  is  notationally 
convenient  to  work  with  the  vector  Lfc(n),  rather  than  with  the  individual 
links,  since  the  decisions  at  each  node  k  depend  on  the  states  of  all  of  the 
possible  outgoing  links.  Lk{n)  might  denote  other  quantities  in  addition  to 
the  channel  quality.  For  example,  there  might  be  power  constraints  that 
vary  randomly  due  to  interference  from  exogenous  sources.  These  could 
be  included  in  the  Lk{n).  If  some  link  at  node  k  is  unavailable  at  time  n, 
then  that  fact  could  also  be  included  in  Lk{n).  For  notational  simplicity, 
we  suppose  that  the  channel  state  vector  takes  only  finitely  many  values 
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for  each  node  k.  The  (vector-valued)  symbol  j  is  used  for  the  canonical 
value  of  L^.{n),  for  any  k,n.  The  range  of  j  will  depend  on  the  node  k, 
but  will  be  suppressed  in  the  notation.  Let  di^k,a{n)  denote  the  number 
of  packets  sent  from  queue  i  of  node  k  to  queue  i  of  node  a  at  time  n.  It 
will  depend  on  the  channel  state  and  the  allocated  resources  (e.g.,  power, 
frequency,  bandwidth).  It  is  always  zero  if  node  k  is  not  on  any  path  for 
source  i.  Let  ai^k{n)  denote  the  actual  random  number  of  arrivals  in  slot  n 
from  the  exterior,  if  any,  from  source  i  at  node  k.  These  will  be  non-zero 
only  for  the  unique  node  k{i)  at  which  source  i  enters  the  network. 

Let  J^n  denote  the  minimal  cr-algebra  that  measures  all  of  the  systems 
data  up  until  time  n  as  well  as  the  channel  states  {L„(fc),A:}  in  slot  n. 
These  channel  states  are  assumed  to  be  available  at  time  n.  Let  denote 
the  expectation  conditioned  on  1F„.  We  say  that  the  packets  sent  in  slot  n 
are  sent  at  time  n,  when  the  scheduling  decisions  are  made. 

Stability:  Definition.  An  appropriate  definition  of  stability  is  a  “uni¬ 
form  mean  recurrence  time”  property.  Suppose  that  there  are  Q  <  <  oo 

and  a  real- valued  function  F(-)  >  0  such  that  the  following  holds:  For  any 
n  and  the  random  time  Ui  =  minjfc  >  n  :  |A(A:)|  <  go})  we  have^ 

En  [cTi  -  n]  <  J^(A(n))/{|x(n)|>,o}-  (2-1) 

Then  the  system  is  said  to  be  stable.  If  |W(n)|  reaches  a  level  gi  >  go)  then 
the  conditional  expectation  of  the  time  required  to  return  to  a  value  go  or 
less  is  bounded  by  a  function  of  gi,  uniformly  in  the  past  history  and  in  n."^ 
Note  that  the  right  side  of  (2.1)  depends  only  on  X{n),  and  nothing  else, 
even  though  there  is  a  conditional  expectation  En  on  the  left  side,  and  the 
channel  and  arrival  processes  are  random  and  correlated. 

The  decision  rule.  The  number  of  packets,  di,fe,a(n),  transmitted  from 
queue  (i,  k)  in  slot  n  to  node  /(t,  k,  a)  depends  on  the  allocated  resources, 
such  as  power,  bandwidth,  or  time.  Such  resources  are  subject  to  con¬ 
straints,  either  locally  (at  each  node)  or  globally  (for  the  entire  network). 
The  constraints  might  be  just  bounds  on  the  total  resources  available  at  a 
node  or  on  the  number  of  packets  than  can  be  sent  in  a  slot,  in  which  case 
the  determination  of  the  di^k,a{'n)  for  all  i,  a  can  be  all  made  at  node  k.  If 
the  constraints  involve  more  than  one  node,  then  making  the  assignments 
requires  coordination  among  the  nodes. 

In  classical  control  theory,  stability  ideas  are  often  used  to  obtain  con¬ 
trols  that  assure  a  stable  system.  Typically,  one  chooses  a  Liapunov  func¬ 
tion  and  then  selects  the  control  that  minimizes  its  “rate  of  change”  on  the 
path.  The  idea  is  similar  in  our  case.  We  will  choose  the  di^k,a{n)  that 

=  oo,  unless  otherwise  defined. 

^This  implies  that  the  sequence  {X{n)}  is  tight  or  bounded  in  probability  (see,  for 
example,  [7,  Theorem  2,  Chapter  6]. 
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minimize  an  approximation  to  EnV{X(n  +  1))  —  V{X{n)).  To  motivate 
what  will  be  done,  let  us  start  with  the  evaluation 


E^XfJn+l)-XfJn) 


—  (u)  j  ^  (^)  E  T  ^  ^i,b(i,k,f3),k(,’^^ 

a  p 

+  terms  of  order  {p  —  2)  in  Xi^k{n). 


Note  that  di,h(i,fc,/3),fe(^)  =  number  of  packets  sent  from  queue  (t,  b{i,  k,  (3)) 
to  queue  (i,  k)  at  time  n.  Hence  the  last  sum  is  the  total  number  of  packets 
arriving  at  node  k  at  time  n  from  all  nodes.  The  sum  over  i,k  of  the  terms 
in  the  second  line  that  do  not  involve  the  at^k  can  be  written  as 


- 

i,k,<x 


'^i,f(i,k,a)Xljpik,a)^n)  di^k^aijl). 


(2.2) 


This  can  be  written  as 


a  /3 


(2.3) 


The  lower  order  terms  in  Xi^k{n)  are  nonlinear  functions  of  the  di^k,a{'n) 
and  higher  conditional  moments  of  the  ai^k{n),  and  would  be  very  hard  to 
deal  with.  It  turns  out,  as  in  [4],  that  is  is  enough  to  base  the  decisions  on 
(2.2)  or  (2.3). 

If  the  decisions  at  node  k  need  not  be  coordinated  with  those  at  any 
other  node,  then  the  decision  is  a  maximizer  in 


max 

{di.fc.Q  ("):*,«} 


i,oc 


^i,k,oc 


(n), 


(2.4) 


subject  to  the  local  constraints.  If  there  are  constraints  that  involve  the 
decisions  at  a  set  of  nodes,  then  the  decisions  for  such  a  set  must  be  made 
together,  and  the  decision  rule  is  a  maximizer  in 


max 

{di  k,a 


^kkXlk\n) 


or,  equivalently,  in 


''^i,fii,k,a)Xfj(i^k,a)  (^) 


di^k^aijd) , 


(2.5) 


max 

{di^k,a  {'fT')'A,k,Oc} 


^  ^  di,k,a{'X)  ^  ^  ^i,b(i,k,f3),k{,X) 

a  fj 


(2.6) 


subject  to  the  constraints.  If  {X(n)}  is  not  a  Markov  process,  then  V{X) 
cannot  be  a  Liapunov  function  for  the  system.  However,  as  shown  in  the 
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next  section,  the  perturbed  Liapunov  function  method  [4,  7,  8]  can  be  used 
to  prove  that  the  maximizing  rules  (2.4),  (2.5),  or  (2.6),  assure  stability 
under  reasonable  conditions. 

Let  Ui^k,a{j,  X)  denote  the  control  function  at  queue  i  at  node  k  for 
the  transmission  to  node  f{i,k,a).  The  Ui^k,a{i,  X)  represents  the  al¬ 
located  resources  (power,  time,  bandwidth,  etc.)  that  are  allocated  at 
queue  (i,  k)  to  the  link  to  node  a.  Also,  unless  otherwise  noted,  its  de¬ 
pendence  on  the  queues  is  only  on  and  the  required  queue  values  at 
the  immediate  upstream  nodes,  namely  the  X^j(^^  k,a)  for  L  o.  If  no 
route  for  source  i  uses  node  k,  then  ignore  Ui^k,a{j,  X).  The  amount  of 
data  that  is  sent  from  queue  (t,  k)  to  queue  i  at  node  a  is  determined  by 
the  allocated  resources  Ui^k,a{j,  X)  and  the  current  channel  state  at  node 
k.  Let  the  function  gi^k,a{j,  Xi^k,Ui,k,a{j,  X))  denote  the  actual  amount 
of  data  that  is  sent  under  current  channel  state  j  and  control  Ui^k,a{')- 
This  defines  di^k,a{n)]  i.e.,  the  channel  rate  for  queue  (i,A:)  on  the  link 
to  node  a,  associated  with  channel  state  j  and  control  Ui^k,a{3,  X{n))  is 
di,k,a{n)  =  gi,k,a{3,  Xi^k{n),Ui^k,a{j,  X{n))).  The  X^^k  appears  as  an  argu¬ 
ment  of  5i,fc,a(')  only  because  the  amount  sent  cannot  be  larger  than  the 
queue  content. 

Assumptions.  The  following  assumptions  are  network  analogs  of  those 
used  in  [4]  and  will  be  commented  on  further  below. 

A2.1.  There  are  constraint  sets  Uk  such  that  {ui^k,a{3,  A);  i,  a}  G  Uk-  It  is 
always  assumed  that  the  maximizing  constrained  di^k,a(ji)  in  (2.4),  (2.5), 
or  (2.6)  exist  and  are  Borel  functions  of  the  {A(n),  Lfe(n);  i,  fc}. 

A2.2.  There  is  a  constant  Ki  such  that  if„|ai(n)|^  <  Ki.  There  are  A“^ 
such  that  the  sums 


^Vt,k{n)  =  Y.  [if„a..fe(0  -  \lk] 

l—n 

converge  as  u  ^  oo,  uniformly  in  n,ui. 

It  follows  from  the  definitions  that  A“^  =  0  if  node  k  is  not  the  source 
node  k{i)  for  source  i.  For  future  use,  write  A“  =  Xlk(i)  mean  input 
rate  for  source  i  (measured  in  packets  per  slot). 

A2.3.  For  each  node  k  there  are  Hkj  >  0  such  that  Iffc j  =  1 

—  life  j]  converges  as  v  —>  oo,  uniformly  in  n,  ui. 

A2.4.  Define  Aq  =  TCLa^i^k,j,u,a,x  bi,fc.a(j,  A^^fe, Mi_fe,a(j,  A))]  .  There  is  a 
resource  allocation  {ui^k,a{‘)',i,k,a\  such  that  the  following  holds  under 
it.  There  are  non-negative  real  numbers  fc,a}  such  that  ql  k  a  ~ 
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(^)  5  (Ji -^(^) )  )  ^  ^0-  AlsO,  if  <C  Kqj 

then  gi^k,a{jjXi^k{ri),Ui^k,a{jjX{n)))  <  q^k^c.-  The  satisfy 

^i,b{i,k,i3),k^Hi^k,/3),j  <  9i,fc  =  Qi,k,cflk,j ,  each  i,k  ^  k{i),  (2.7) 

/3j  i.“ 

and,  for  k  =  k{i),  Xf  <  Qi^k- 


Comments  on  the  assumptions.  (A2.1)  simply  states  that  there  are 
constraints  on  the  resources  and  allocations.  (A2.2)  and  (A2.3)  are  mixing 
conditions  on  the  data  arrival  and  channel  processes,  resp.,  and  do  not 
appear  to  be  restrictive.  If  the  arrivals  occur  in  batches,  with  the  batches 
and  intervals  being  mutually  independent  and  each  iid,  then  (A2.2)  is  just 
a  constant  times  the  residual  time  to  the  first  arrival.  See  [4]  for  more 
discussion  of  this  point.  Both  (A2.2)  and  (A2.3)  say  that  the  expectation  of 
the  future  values  of  the  random  variables  given  the  data  in  the  remote  past 
converges  to  the  average  in  a  “summable”  way  as  the  difference  between  the 
times  goes  to  infinity.  (A2.3)  holds  for  the  received  signal  power  associated 
with  Rayleigh  fading.  (A2.4)  basically  requires  that  there  are  controls 
under  which  the  mean  service  rate  at  queue  (i,  k)  for  any  i  that  uses  node 
k  is  slightly  greater  than  the  mean  data  arrival  rate  A“,  if  the  queues  remain 
large,  for  all  (z,  k).  Similar  conditions  occur  frequently  in  studies  of  stability 
in  stochastic  networks. 

A  variation  of  (A2.2)  and  (A2.3).  The  convergence  of  the  sum  in 
(A2.2)  can  be  replaced  by  the  condition  that  Enai^k{l)  —  X^  f.  — >  0  uniformly 
in  n,  w  as  fc  —  n  ^  oo.  Then  the  perturbation  (3.1)  would  be  replaced  by 

m+n 

'^i,k^f^k  (^)  [ai,fc(0  ~  \,k] 

l—n 

for  large  enough  m.  The  error  terms  in  the  proof  are  slightly  different, 
but  the  method  is  the  same.  Analogous  remarks  hold  for  (A2.3)  and  the 
perturbations  (3.2). 

An  eqnivalent  form  of  (A2.4).  Abusing  terminology  slightly,  for  k  yf 
k{i),  define  qiXi,k)  =  J2/3,j  ^ib{i,k,f3),k^bii,k,l3),j^  the  average  (over  the  chan¬ 
nel  variations)  fiow  into  (z,fc)  under  the  rates  {qlf.^;i,k,j,a}.  Then  it  is 
implied  by  (A2.4)  that  the  9- j.  „  can  be  taken  to  satisfy 

\  )  (2.8a) 

^The  lower  bound  Kq  is  introduced  in  (A2.4)  only  because  if  the  queue  content  is 
smaller  than  the  maximum  of  what  can  be  transmitted  on  a  scheduling  interval,  then  the 
mean  (weighed  with  the  j)  output  might  be  too  small  to  assure  (2.7).  For  example 
if  a  queue  is  empty,  then  there  are  no  departures. 
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and  that  there  is  cq  >  0  such  that  for  k  ^  k{i), 

average  into  (i, /c)— average  out  of  {i,k)  =  qi^b(i,k)~Qi,k  <  —cq  <  0.  (2.8b) 

Section  5  gives  a  useful  method  for  getting  both  the  routing  and  the  qj  a- 

Example.  Let  the  control  be  over  bandwidth,  with  the  rate  proportional 
to  bandwidth.  Let  the  bandwidth  allocated  to  {i,  k)  for  transmission  to 
node  a  be  denoted  by  Bf  let  the  constants  of  proportionality  be  ^  ^ 

and  define  the  rate  qlk  a  ~  ^ik  a^l  k  a  -  There  are  the  total  bandwidth  con¬ 
straints  ^  ■  a^ik  a  —  for  each  j,  k.  Suppose  that  the  set  of  inequalities 

qi,k,a^k,j  >  A“,  all  k,  has  a  solution.  Then  the  corresponding 
satisfy  ’(A2.4). 

3.  The  Stability  Theorem  and  Liapunov  Function  Perturba¬ 
tions.  Suppose  that,  for  a  random  process  {a;(n)},  we  have  EnV{x{n  + 
1))  —  V{x{n))  =  Cn,  where  {c„}  is  a  random  sequence  that  is  “mixing”  in 
the  following  sense.  There  is  a  constant  c  <  0  such  that  —  c]  ^  0 

fast  enough  as  to  ^  oo,  for  the  sum  SVn  =  J2i=n^n[ci  —  c]  to  converge 
(and  be  bounded)  uniformly  in  n,  where  E„  now  denotes  the  expecta¬ 
tion  conditioned  on  {ci,l  <  n}.  Define  =  V(x(nA))  +  SVn-  Then 
Bn^Vn+i  c)  and  [c^  c]  —  c  <  0. 

The  use  of  the  perturbation  has  allowed  us  to  replace  c„  by  a  “mean.”  The 
perturbed  Liapunov  function  method  is  an  extension  of  this  idea. 

The  perturbation  5V (n)  that  will  be  used  will  be  a  sum  of  components, 
one  associated  with  each  possible  external  input  process,  and  one  associated 
with  each  input  link  and  one  to  each  output  link  of  each  queue.  The 
motivation  for  their  form  should  be  apparent  from  the  way  that  they  are 
used  in  the  proof.  See  also  [5,  7]  for  more  motivation  of  the  construction 
of  the  perturbations.  Recall  that  k{i)  denotes  the  arrival  node  for  source 
i.  The  perturbation  associated  with  the  arrivals  from  source  i  is 


<5y,“fc(n)  =  ^  [a.^l)  -  .  (3.1) 

l—n 


This  is  zero  \i  k  ^  k{i) . 

The  function  ^(n)  defined  in  (3.2)  is  concerned  with  the  effects 

of  the  departure  of  packets  from  queue  (z,  fc),  via  link  f{i,k,a),  on  the 
value  of  EnXfj,{n+  1)  —  X^ j,{n)  when  j  is  the  vector- valued  channel  state 
at  node  fc,  and  under  the  fixed  rate  qj  j,  ^  defined  in  (A2.4).  The  SVfjTj  ^{11) 
is  concerned  with  the  effects  on  EnXff.  {n+l)-XP  (n)  of  the  inputs  to 
(z,  k)  from  the  link  leading  to  it  from  node  b(i,  k,  (3),  when  the  vector-valued 
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Theorem  3.1.  The  system  is  stable  under  (A2.1)-(A2.4). 

Proof.  The  function  V (n)  is  the  (time- varying)  Liapunov  function  that  is 
to  be  used.  We  need  to  show  that  V (n)  is  a  local  supermartingale,  when 
the  queue  values  are  large.  In  particular,  we  need  to  show  that  there  is 
c  >  0  such  that  for  large  X,  we  have  EnV{n  -I-  1)  —  ^(n)  <  — c,  and  then 
to  show  that  this  inequality  can  be  used  to  get  (2.1).  Thus,  we  need  to 
evaluate 

EnV{n+  1)  -V{n)  =  '^Wi^kEn  Xfk{n+  1)  -  Xfj^{n) 

i,k 

+  ^  [(5IL“fc(n  -I-  1)  -  5Pj“j,(n)] 

i^k 

i,k,j,oc 

+  E  <5^*E./3(^+i)-'^ECi,/3(”)  ■ 

The  components  will  be  evaluated  separately,  and  then  the  results  summed. 
The  summation  will  effectively  cancel  various  “undesirable”  terms,  and 
replace  them  by  averages.  This  is  the  key  idea  of  the  method.  In  the 
expansions  to  follow,  K  denotes  a  constant  whose  value  might  vary  from 
usage  to  usage.  A  first  order  Taylor  expansion  yields 

A(^fc(n+ 1)  -  A(^fc(n)  = 

i^k 

^  ^  (^)  ^  ^  dj^k.Oi  ('^)  ^  ^  (^) 

i,k  ol  /3 

+0{\XP-^{n)\)  +  K. 
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Now  consider  (3.1)  for  any  i,k.  Recall  that  SV^\{n)  =  0  if  A:  yf  k{i),  the 
origin  node  for  source  i.  If  k  =  k(i),  then  a  first  order  expansion  yields 

E„SV-k{n+l)-SV::,{n) 

=  -m,kXf-\n)  +  0{\XP-^{n)\)  +  K. 

Thus, 


Y,En[SV::,{n  +  l)-SV,%n)] 

i,k 

=  -J2w,,kX!-\n)  [E^a,Mn)-\l,]+0{\XP-^{n)\)  +  K. 

i,k 

(3.5) 

Let  us  see  what  has  been  accomplished  so  far.  On  adding  (3.4)  and  (3.5), 
we  see  that  the  terms  Wi^kXf'^^ {n)Enai^k{Ti)  are  cancelled,  and  a  “mean 
value”  term  Wi^kXf^^{n)Xfj^  appears,  together  with  a  term  of  order  p—2. 
These  lower  order  terms  will  be  dominated  by  the  terms  of  order  p  —  1 
for  large  values  of  the  queue  state.  The  desire  for  such  cancellations  and 
replacements  by  mean  values  determined  the  form  of  (3.1). 

Let  us  now  consider  the  perturbation  defined  by  the  first  term  in  (3.2), 
which  will  facilitate  dominating  the  di^k,a{n)  in  (3.4)  by  a  term  that  can 
be  effectively  averaged.  The  definition  (3.2)  yields 


En 


i,k,j,cx  V 


Wi^kEnXfj,  En+l  [l{Lk{l)=j}  ^k,j\ 

/— n+1 
oo 


l—n 


(3.6) 

Rewrite  (3.6)  by  splitting  out  the  lowest  summand  of  the  last  term  to  get 

OO 

—  Wi^kEnXfj.  {'n+f)qi^k,a  ^  En+l\l{Lk(l)=j}  ~^k,j\ 
oo 

+Wi,kXl^^{n)qlj^^^  L;„  -Hfej]  . 


(3.7) 


By  expanding  Xff,^{n  +  1)  —  Xfi^^{n)  we  can  represent  (3.7)  as 


i,k 


En 


=  ^^,kXYi^^,k,.  [hLkin)=,}  -  n^,,]  +  o{\x^-\n)\)  +  K. 


(3.8) 
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An  analogous  procedure  yields  that 


E, 


+0{\XP-^{n)\)  +  K. 


^{Lb(i,k,i3}{n)-j}  ^b{i,k,f3),j 


(3.9) 


Now  add  the  expansions  (3.4),  (3.5),  (3.8),  and  (3.9),  over 
Some  terms  in  one  expansion  are  the  negative  of  terms  in  some  other  ex¬ 
pansions.  Adding  the  expansions  and  canceling  such  terms  yields  the  ex¬ 
pression 


EnV{n+l)  -  V{n)  =  ^ 

i^k 

+  ^  —  Wi^k  di^k,aij^^  “1“  (^) 

i,k  ^  oc  (3  ■ 

+  w;,,feAP"\n)  Y  Ak,c.  [hLk(n)=j}  -  Hfej] 

i,k,j  ct 

~  E  E  ^lb(i,fe,/3),fc  hLb(i,k,p)(.n)=j}  ~  ^b{i,k,l3),j 

+0(\XP-^{n)\)  + 

(3.10) 

The  terms  in  the  second,  third  and  fourth  lines  of  (3.10)  that  do  not  involve 
the  Ilfc  j  variables  are 


^  )  ^i,kX^  {n)  ^  (  dj^k.a  ijE)  ^  'Ak,b{i,k,0),k(,A 

i,k  ^  a  j3 

+  E^*''=Efc  E  (")=!}  ~  E  ^i.h(i..k.a^A\Lb(i.k.a',{n)=j} 

i,k  a,j  13, j 


For  each  k,  a,  j3,  the  indicator  functions  in  the  above  sums  over  j  select  the 
actual  current  channel  state  j  =  Lk{n)  or  j  =  Lb(i,k,(3)in)i  appropriate. 
Hence,  the  previous  expression  can  be  rewritten  as 


^  ^  '^i,kX^ (n)  ^  (  di^k,a{A  ^  )  ^i,b(i,k,^),k(,‘kl) 

i^k  *"  Q:  0 

V  «;■  i.XP~^(n\  \  V  -  V  A>‘^kk,p)« 

+  /^Wi^k^i^k  Z_^1i,k,a  /  Xi.hd.k.Fi^.k 

i,k  *-  Q:  (3 


(3.11) 


It  is  simpler  to  complete  the  proof  first  under  the  assumption  that 
Xi,k{n)  >  Kq  for  all  i,k,  and  then  to  add  the  few  details  for  the  general 
case.  If  all  XiAA  >  Kq  then  by  (A2.4)  there  are  resource  allocations 
{ui,k,a{')'^  b  A  such  that,  for  channel  state  j,  the  output  from  queue  {i,  k) 
to  queue  {i,f{i,k,a))  will  be  gi,fc,a(j,  Ai,fe(n),  Mi,fc.a(j,  A(n)))  =  The 
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di^k,a{n)  are  chosen  by  either  the  maximization  rule  (2.4),  or  by  the  rules 
(2.5)  or  (2.6)  (which  are  equivalent  to  each  other).  The  rule  (2.4)  is  implied 
(2.5)  and  by  (2.6).  On  the  other  hand,  the  qf  ^  defined  in  (A2.4)  are  not 
necessarily  maximizers  in  (2.6).  Hence  the  expression  (3.11)  is  non-positive. 
Using  this  non-positivity  in  (3.10)  together  with  the  definitions  of  qi^k,a, 
(ji^k  and  qi^b(i,k)  yields  the  following  upper  bound  to  (3.10): 

''^Wi^kXfk^{n)[X°lj^  —  qi^k  +  qi,b{i,k)]+0{\X^  ‘^{n)\)  +  K.  (3.12) 

By  (2.8),  the  terms  in  the  brackets  in  the  first  line  of  (3.12)  are  <  — cq  <  0. 
Thus  we  have  proved  that 

U„U(n+  1)  -  U(n)  <  -co^u;.,fcAPfe^(n)  +  0{\X{n)\P-^)  +  K,  (3.13) 

i,k 


6V{n)  satisfies 

\6V{n)\  =  0{\X{n)\P-^)  +  K  (3.14) 

and,  by  (3.13), 

EnV{n+  1)  —  V{n)  — oo,  uniformly  in  n,w  as  X{n)  oo.  (3.15) 

By  (3.15),  there  are  ci  >  0  and  qo  >  0,  such  that,  for  |A(n)|  >  qo, 

E„V{n+l)  -V{n)  <-ci.  (3.16) 

Given  small  5  >  0,  (3.14)  implies  that  for  go  sufficiently  large, 

\V{X{n))  -  V{n)\  <  5(1  -k  V{X{n)).  (3.17) 

Let  (To  be  a  stopping  time  for  which  |A((Jo)|  =  C2  >  go,  and  define  the 
stopping  time  cti  =  minjn  >  cto  :  1^(71)!  <  go}.  Then,  by  (3.16),  we  have 

EaoVio-i)  -  V{ao)  <  -ciEaoiui  -  cto].  (3.18) 

Using  (3.18)  and  the  bound  (3.17)  on  U(n)  —  V{X{n))  to  bound  U((Ti)  — 
V{X{ai)),  f  =  0, 1,  yields 

-SE^^  [1  +  V{X{ai))]  +  E,„V{X{ai)) 

<  E„gV{ai)  <  -ciEao{(Ji  -  (To)  -I-  [5-1-  U(A(cto))(1  -I-  5)] 
or 

^  ^  /  25  +  U(A((To))(1  +  5)  +  SE,,V{X{a^)) 

Ecro\<Jl  —  (To)  <  - 

which  implies  that  the  definition  of  stability  (2.1)  holds  since  V{X{ai))  < 
suP|^|<,o  U(x). 
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Finally,  we  complete  the  details  when  some  components  of  X(n)  are 
less  than  Kq.  Recall  the  definition  of  uj  f,  ^(•)  and  qh.  ^  in  (A2.4).  Define 

For  X^^k{n)  >  Kq,  we  have  gi,k,a{Lk{n),  X{n))  =  by  the  defi¬ 
nition  of  the  in  (A2.4).  If  Xi^k{n)  <  Kq  then,  also  by  (A2.4), 

9i,k,aiLk{n),X{n))  <  ■  Rewrite  (3.11)  as  follows. 


-  ^  Wi^kXl^^{n)  ^  d^^kA'n)  -  X!  d^M^,k,0),k{'^) 

i,k  a  (3 


i,k 


^  ^  9i,b(i,k,f3),k{^b{i,k,0)  ('^)  7  -^(^)) 


+  w,^kXl,^^{n)J2  %'k^^  -9i,kALk{n),X{n)) 

i,k:Xi^i~{n)'>KQ  ot 

+  ^  w^^kX'^~^{n)  -J29^Xc^(^k{n),X{n)) 

i,k:Xi^k{n)<.KQ  gl  a. 

-  w,,fcAP"^(n)  Y  in^Yfihk  -  J2  9^Mi,k,0),k{Lb{^,KP){n),X{n)) 
L  /3  /3 


i.k 


As  was  argued  for  the  case  where  all  >  Kq,  the  sum  of  the  first  three 
lines  is  non-positive,  since  the  b  chosen  by  the  maximiza¬ 

tion  rule.  The  two  terms  in  each  bracket  in  the  fourth  line  are  equal  by 
the  definition  of  the  gi^k,a  when  Xi^k{n)  >  Kq.  Hence  this  term  is  zero. 
By  (A2.4),  the  bracketed  terms  in  the  last  line  are  non-negative,  hence  the 
last  line  is  non-positive.  Thus  the  only  possible  positive  term  is  the  next  to 
last  line,  and  this  is  0(1)  since  it  is  a  sum  over  i,  k  for  which  Xi^k{n)  <  Kq. 
Thus  (3.11)  is  0(1).  From  this  point  on  the  proof  is  completed  just  as  for 
the  case  where  all  Xi^k{n)  >  Kq.  M 


Notes.  The  decision  rule  (2.4)  requires  that  each  node  k  know  the  value 
of  the  Xi^k{n)  and  Xij(^i^ka)(ki)  for  all  i,a  that  are  relevant  at  node  k.  In 
fact,  if  the  value  of  the  Xij(^i  k,a){n)  were  known  only  subject  to  a  bounded 
error,  then  the  proof  still  goes  through  under  the  same  conditions.  So,  only 
an  occasional  approximate  estimate  of  the  queues  at  the  upstream  nodes  is 
needed.  Suppose  that  some  links  are  preempted  by  priority  users  from  time 
to  time,  where  the  intervals  of  availability  are  defined  by  a  renewal  process 
that  is  independent  of  the  arrival  and  channel  rate  processes.  Then  it  can 
be  shown  that  the  results  continue  to  hold,  but  with  the  qi^k  multiplied 
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by  the  fraction  of  time  that  the  channel  is  available,  so  the  capacity  must 
be  sufficient  to  handle  the  average  down  times.  Under  the  other  assump¬ 
tions,  condition  (A2.4)  is  sufficient  but  not  necessary  for  stability.  But  it  is 
“nearly”  necessary  in  the  sense  that  if  for  each  choice  of  the  {9^  q,},  there 
is  some  (to,  kg)  such  that  ~  9*o.fco  >  then  the  system  would  not 

be  stable. 


4.  Some  Extensions.  The  basic  approach  to  scheduling  and  stability 
can  be  extended  in  many  ways,  and  the  examples  described  below  illustrate 
some  of  the  possibilities. 

A.  Acknowledgments  of  receipt  required  for  each  link.  The  fore¬ 
going  development  did  not  require  that  received  packets  be  acknowledged. 
Suppose  that  packets  on  the  link  from  any  queue  (z,  k)  to  node  a  that  are 
not  acknowledged  within  a  window  Wi^k,a  of  scheduling  intervals  will  need 
to  be  requeued  at  (z,  k)  and  retransmitted.  The  treatment  of  the  acknowl¬ 
edgment  and  loss  processes  involves  a  more  complicated  notation  and  an 
additional  perturbation  to  the  Liapunov  function.  In  order  to  keep  the 
notation  reasonable,  we  will  suppose  that  the  routing  is  unique  for  each 
source-destination  pair.  Thus,  the  indices  a,  [3  can  and  will  be  dropped. 
The  approach  for  the  non-unique  routing  case  is  essentially  the  same,  with 
analogous  results.  The  acks  are  sent  back  to  the  previous  node  when  a 
packet  is  received,  subject  to  a  possible  delay. 

If  we  fully  accounted  for  the  possibility  that  the  packet  loss  or  non-ack 
process  depended  on  the  traffic  in  the  channel,  and  the  channel  charac¬ 
teristics,  the  resulting  problem  would  be  very  difficult.  Because  of  this,  it 
is  often  assumed  that  the  loss  is  a  consequence  of  uncontrolled  additional 
traffic  in  the  channels.  We  will  take  the  following  often  used  approach.  For 
each  link,  an  ack  for  each  received  packet  is  sent  to  the  node  from  which 
it  just  came.  If  a  packet  sent  from  queue  (z,  k)  at  time  n  is  not  acknowl¬ 
edged  by  time  n  +  Wi^k,  then  that  packet  will  be  requeued  at  (z,  k).  The 
development  in  Section  3  can  be  readily  modified  to  accommodate  these 
changes.  The  development  in  [6]  supposed  that  acks  for  source  i  data  are 
sent  only  to  the  origin  node  k{i),  and  that  packets  lost  anywhere  must  be 
retransmitted  from  that  node.  Here  acks  are  required  for  each  link. 

Until  the  end  of  the  example,  we  suppose  that  the  packet  loss  process 
is  random.  Thus,  the  events  that  packets  are  lost  are  independent  among 
the  inks,  iid  for  the  packets  on  each  link,  and  independent  of  the  channel 
states,  decisions,  and  arrivals.  Let  Ci,k{n)  denote  the  fraction  of  packets 
sent  from  queue  (z,  k)  at  time  n  that  were  not  received  at  queue  (z,  /(z,  k)). 
These  would  not  be  acknowledged  by  the  end  of  the  waiting  period  Wi^k, 
and  must  be  requeued  and  retransmitted  at  that  time.  Let  Tn  now  measure 
the  <  n  for  all  i,k  ,as  well.  Define  pi^k  =  EnCi,k{n)  =  EQ^kin). 
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The  queue  dynamics  are  now 


The  last  term  on  the  right  are  the  requeued  packets,  and  the  next  to  last 
term  are  the  packets  sent  from  {i,b{i,k))  to  {i,k)  that  were  received.  We 
have 


Xlki^+^)  = 

^ik^in)[- di^k{n)  +  {1  -  Q^i,{x,k){n))d,^b{i,k){n)  ^  ^ 

(4.1) 

~kdi^k{'kl  ^^i,k)Ci,k{kl 

+XP-\n)a,,k{n)  +  0{\XP-\n)\)  +  K, 

where  X  is  a  constant  whose  value  might  change  from  usage  to  usage.  The 
additional  Liapunov  function  perturbation  component 

n—1 

6V^{n)  =  J2w^,kXll\n)  Ak{l)Q,k{l)-  (4-2) 


will  help  us  deal  with  averaging  the  increases  in  the  various  queues  due  to 
not  receiving  an  ack  in  time. 

Recall  that  if  fc  =  k{i),  the  origin  node  for  source  i,  then  dim^k){‘fi)  = 
0.  Noting  that,  for  k  yf  k{i),  i?nCi,6(i,/c)(^)  =  Pi,b(i,k)j  we  can  write 


E4V{X{n  +  1))  -  V{X{n))]  +  E^SV^in  +  1)  -  5V^ {n)\ 


=  y^Wj^kXf {n)Enai,k{n) 

i,k 

+  '^Wi,kXll^{n)[  -  d^^k{n)  +  {1  -  p^Xi,k))d^Xi,k)('^) 

i,k 


+E 


Pi^kdi^k{n) 


+0{\XP-^{n)\)  +  K. 


li,k{n  -  Wi^k)di^k{n  -  Wi^k) 


(4.3) 

The  second,  third  and  fourth  lines  contain  the  highest  order  terms  in 
EnV{X{n  +  1))  —  V{X{n)),  and  the  next  to  last  line  is  the  highest  or¬ 
der  term  in  the  expansion  of  En[SV^ (n  -I-  1)  —  5V^ {n)].  The  terms  with 
di^kin  —  Wi^k)  lines  cancel  each  other,  and  we  drop  them  now. 

The  decision  rule  that  replaces  (2.4)  is 


max 


E 


(l-p*,fc)  m,kXl^^{n) -Wij,^i^k)Xfj^^,,^{ri)  d^^k{n). 


16 


The  rules  (2.5),  and  (2.6)  are  modified  similarly.  The  full  new  perturbed 
Liapunov  function  is 


V^{n)  =  V{X{n))  +  SV^in)  +  ^  <5y,“fc(n)  +  ^(1 

i,k  i,k 

+  -  PiM^,k)W^k~jin)- 

i,k,j 


Then,  using  (4.1),  (4.3),  (3.5),  (3.8),  and  (3.9), 


(4.4) 


E^V^in  +  1)  -  V^{n)  =  ^ 

i,k 

+  X!  [“  -  Pi,k)wi^kXfj:^ {n)d^^kin) 

i,k 

+  (1  Pi,b(i,k))'^i,k^ii^  (^) 

+  ^~~l(l  ~  Pi,k)Wi^kXf  f.  {kl)qjj^  [kkk{n)=j}  ~  n^j] 


i.k 


-  ^(1 

+o(\XP-^{n)\)  +  K. 


kkb{i,k)(rt.)=j}  ^b{i,k),j 


The  second  and  third  lines  are  due  to  (the  non-arrival  parts  of)  the  third- 
fifth  lines  of  (4.3).  The  fourth  line  is  due  to  (1  —  Pi,k)dVfl^j{n)  and  the 

next  to  last  line  to  (1  —Pi^b(i,k))^Vfk~j{n).  Dominating  terms  as  in  the  part 
of  the  proof  of  the  theorem  concerning  (3.11)  yields  the  following  upper 
bound  to  the  last  expression: 

^  (  ^i,kX^ {ti)  (1  Pi,k)Qi,k  “t”  (1  Pi,b{i,k))Qi,b(i,k)\ 

Sj{\XP-\n)\)  +  K. 


At  most  one  of  qi^b(i,k)  and  A“^  can  be  non-zero  for  any  i,  k. 

The  condition  (A2.4)  is  modified  to  read,  for  all  i, 

(1  -  Pi,k)qi,k  >  Ai,  for  k  =  k{i), 

(1  -  Pi,k)qb,k  -  (1  -  Pt,b(t,k)dt,b(^,k)  >  0,  for  k  ^  k{i). 

The  proof  is  completed  as  in  Theorem  3.1. 

If  the  packet  loss  process  for  the  link  out  of  {i,  k)  is  correlated,  then 
the  process  {Ci_/c(n),n}  is  correlated  and  another  perturbation  is  required 
to  average  it.  Suppose  that  there  are  pi^k  such  that  the  sums  in 


6V^^+{n)  =  w,,kXl 


(n)  E„ 

l—n 


Ci,fc(0  Pi,k  ^2,^(0? 
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oo 

(^)  “  (^)  ^  Ci,b(i,k){^^  Pi,b{i^k)  ^i,6(i,fc)(0 


are  well  defined  and  bounded,  uniformly  in  n,uj.  Then  add  the 
to  V^(-).  The  conclusion  is  unchanged. 


B.  Multicasting.  Suppose  that  some  sources  have  multiple  destinations, 
with  a  unique  route  for  each  source-destination  pair.  Let  the  route  network 
for  each  source  form  a  tree,  with  the  source  as  the  root  and  the  final 
destinations  as  the  end  branches.  Suppose  that  if  the  tree  branches  at  node 
k,  then  transmissions  must  be  done  to  all  of  the  branches  simultaneously, 
as  is  commonly  required  in  multicasting.  If  the  route  for  source  i  uses  node 
k,  then  redefine  b{i,  k,  7)  to  denote  the  nodes  at  the  end  of  the  branches  of 
the  tree  out  of  queue  {i,k),  where  the  dimension  of  the  index  parameter  7 
is  the  number  of  branches. 

Then  (2.4)  is  replaced  by 


max  y 

{di,k{n):i} 


7 


(n) , 


subject  to  the  constraints  at  node  k.  Modify  (2.5)  and  (2.6)  analogously. 
The  criterion  (A2.4)  is  modified  in  an  obvious  manner  to  take  account  of 
the  new  flows. 


C.  Variable  number  of  sources  and  destinations.  When  the  number 
of  sources,  nodes  and  destinations  vary  randomly,  the  modeling  problem 
can  be  quite  vexing.  For  example,  if  a  node  disappears  slowly  as  its  links 
fade,  what  happens  to  its  still  untransmitted  data  ?  We  will  take  a  simple 
approach,  by  supposing  that  there  is  a  backbone  network,  with  an  un¬ 
changing  number  of  nodes,  although  the  associated  links  in  the  backbone 
will  still  vary  randomly.  There  is  a  large  and  randomly  varying  number  of 
sources  that  send  data  to  the  nodes  in  the  backbone.  The  arriving  packets 
from  the  randomly  changing  number  of  sources  are  multiplexed  on  arrival. 
These  packets  are  assigned  priority  values  and  at  the  backbone  nodes,  the 
data  is  queued  according  to  both  priority  and  the  node  to  which  that  packet 
would  be  sent  to  next  on  its  route  to  its  final  destination. 

Owing  to  the  multiplexing  and  the  large  number  of  sources,  it  is  as¬ 
sumed  that  the  total  arrival  processes  (per  slot)  from  the  exterior  to  the 
various  queues  (i,  k)  are  mutually  independent,  and  the  elements  of  each 
are  iid,  with  bounded  variances,  and  means  denoted  by  A“j,.  The  index  i 
denotes  the  ith  queue  at  backbone  node  k,  and  that  queue  is  associated 
with  both  priority  and  the  next  node,  and  might  contain  packets  from 
many  different  sources.  Let  ^i^k\v,-y{ji)  denote  the  fraction  of  the  number 
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of  packets  that  are  sent  at  time  n  from  queue  (t,  k)  to  node  7  will  be  as¬ 
signed  to  queue  v  there.  Again,  owing  to  the  multiplexing  and  the  large 
number  of  sources,  we  suppose  a  conditional  independence  in  routing  in 
that  there  are  Pi,k;v,'y  such  that  En(,i,k-,v,-y{'n)  =  Pi,k-,v,'y  ,  where  denotes 
the  expectation  conditioned  on  the  data  to  time  n. 

The  queue  dynamics  are 

X^,k{n  -I-  1)  =  Xi^k{n)  +  Qi^kin)  -  d^^k{n)  +  ^ 

Condition  (A2.4)  is  changed  to  require  the  existence  of  {qf  such 

that 

\,k  +  di,^Pv,'y:i,k^^,j  ~  df^k^kj  <  0 

j 

for  each  i,k.  The  proof  follows  the  lines  of  that  of  Theorem  3.1.  The 
decision  rule  (2.4)  is  replaced  by,  for  each  node  k  and  channel  state  j, 


max 

{di.k'yi} 


i: 


XfpM-Y.Xfpp’,,,,. 

v^'y 


di,k{n). 


5.  An  A  Priori  Routing  Selection.  A  potentially  useful  approach 
for  getting  the  routing  and  the  «.(•)  functions  is  based  on  a  type  of  fluid 
controlled-flow  approximation.  In  applications  the  algorithm  would  be  run 
periodically  to  produce  new  routings  as  conditions  change.  The  example 
is  intended  to  be  illustrative  of  the  possibilities  only.  Suppose  that  power 
only  is  to  be  allocated.  Let  pi  f.  ^  denote  the  power  assigned  to  queue  (i,  k) 
for  data  transmitted  to  node  a,  when  the  channel  state  at  node  k  is  j.  The 
associated  channel  rate  is  Cf  j.  ^{pl  f.  The  routes  to  be  given 

might  depend  on  the  channel  states.  But  the  development  in  Section  3  is 
readily  modified  to  account  for  this  dependence. 

Suppose  that  there  are  upper  bounds  Qi  such  that  for  each  qj. 


^Clka{p>^,kJ<Qk-  (5.1) 

2, a 


This  might  reflects  the  fact  that  each  packet  takes  a  minimal  time.  Suppose 
that  each  node  k  has  a  constraint  of  the  form 

J2pik,c.<Pk,  eachj,  (5.2) 

i,cx 


where  Pk  is  the  total  energy/slot  available  at  node  k.  We  also  need  a 
constraint  that  assures  that  the  average  output  for  each  non-source  node 
equals  the  average  input,  and  we  write  this  as  follows,  for  each  i,k  ^  k{i)\ 

^  C'*.fe.a(plfc.a)nfcj  >  XI  =  ^-  (5.3) 
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If  node  k{i)  is  the  input  node  for  source  i,  then  replace  (5.3)  by 


out  =  J2CI 


(v^ 

k{i)  ,a  ^^i^k{i)  ,q 


)n 


Hi), 3 


=  A? 


(5.4) 


The  (arbitrarily  small)  e  >  0  is  used  to  assure  slight  overcapacity  so  that 
(A2.4)  will  hold  and  the  stability  argument  of  Theorem  3.1  can  be  used. 
Suppose  that  c{i)  is  the  destination  node  for  source  i.  Then  to  assure  that 
all  packets  end  up  where  they  are  intended,  for  each  i  use  the  constraint 


^  k  rd^iPj  k  ~  Aj  +  I 


k,3 


(5.5) 


ilk, a  =  ^i,k,a(pik,a)  ^^at  Satisfy  the  constraints  (5.1)-(5.5)  will 
yield  an  acceptable  a  priori  route.  But  one  might  wish  to  select  one  via 
an  optimization  problem.  One  possible  cost  criterion  is  the  total  average 
power  given  by 


(5.6) 


Minimize  (5.6),  subject  to  (5.1)-(5.5).  The  above  approach  to  getting  the 
a  priori  routes  might  yield  a  distributed  flow  for  some  sources.  However, 
given  these  routes,  the  maximization  rules  (2.4),  (2.5),  or  (2.6),  still  work. 
Replace  (2.4)  by 


max 

{di.fc.Q  (");».«} 


vP—^ 

^i,fU,i,k,a) 


i^i,k,a  (kl) , 


where  for  each  i,j,  k,  f{j,  i,  k,  a)  indexes  the  links  for  which  Pi  ^  ^  ^ 
di,k,a{n)  is  the  amount  sent  to  node  a  from  queue  (i,  k). 

For  multicasting,  use  (5.5)  for  all  destination  nodes  for  source  i. 

The  criterion  (5.6)  is  concerned  with  total  power.  An  alternative  is  to 
strive  for  maximum  stability.  To  do  this  rewrite  (5.3)  as 


1,3 


where  >  0.  With  appropriate  definitions,  this  can  be  made  to  include 
(5.3)  and  (5.4).  Then  either  maximize  /c  seek  maxmini  febi  fc. 

This  approach  will  get  routes  and  qh^  ^  that  yield  the  best  cq  in  (A2.4).  In 
addition,  the  dual  variables  associated  with  the  constraints  provide  “price” 
guidelines,  that  tell  us  the  places  where  an  increase  in  the  resources  would 
do  the  most  good  (in  the  sense  of  the  mathematical  programming  formu¬ 
lation)  . 

The  example  in  [6,  Section  5]  was  concerned  with  a  simpler  model, 
where  each  packet  that  was  transmitted  was  required  to  have  a  minimum 
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S/N  ratio  at  the  receiver,  and  the  final  form  of  the  optimization  problem 
was  a  linear  program. 

Comment  on  another  case:  bandwidth  allocation.  Suppose  that  the 
basic  control  is  over  bandwidth  allocation,  with  the  number  of  packets/slot 
being  proportional  to  bandwidth  as  ^  =  H,k,aPi,k,a’  where  the  ^ 
are  the  constants  of  proportionality  and  q,  is  the  assigned  bandwidth. 
There  would  be  a  total  BW  constraint  of  the  form  ^  ^  <  i?fc  at  each 

node,  replacing  (5.2).  Input-output  constraints  analogous  to  (5.3),  (5.4), 
and  (5.5),  are  still  to  hold.  To  get  the  routes,  one  could  either  strive  for 
maximum  stability  or  minimize  the  total  average  bandwidth,  which  is 

E  H,k,Ji^,r 

i,k,cx,j 
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